Understanding covariate shift in model performance [ version
نویسندگان
چکیده
Three (3) different methods (logistic regression, covariate shift and k-NN) were applied to five (5) internal datasets and one (1) external, publically available dataset where covariate shift existed. In all cases, k-NN’s performance was inferior to either logistic regression or covariate shift. Surprisingly, there was no obvious advantage for using covariate shift to reweight the training data in the examined datasets. This article is included in the Chemical information channel. science Brian Goldman ( ) Corresponding author: [email protected] McGaughey G, Walters WP and Goldman B. How to cite this article: Understanding covariate shift in model performance [version 1; 2016, (Chem Inf Sci):597 (doi: ) referees: 2 approved with reservations] F1000Research 5 10.12688/f1000research.8317.1 © 2016 McGaughey G . This is an open access article distributed under the terms of the , Copyright: et al Creative Commons Attribution Licence which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the (CC0 1.0 Public domain dedication). Creative Commons Zero "No rights reserved" data waiver The author(s) declared that no grants were involved in supporting this work. Grant information: Competing interests: No competing interests were disclosed. 07 Apr 2016, (Chem Inf Sci):597 (doi: ) First published: 5 10.12688/f1000research.8317.1 Referee Status:
منابع مشابه
Understanding covariate shift in model performance
Three (3) different methods (logistic regression, covariate shift and k-NN) were applied to five (5) internal datasets and one (1) external, publically available dataset where covariate shift existed. In all cases, k-NN's performance was inferior to either logistic regression or covariate shift. Surprisingly, there was no obvious advantage for using covariate shift to reweight the training data...
متن کاملA Probabilistic Covariate Shift Assumption for Domain Adaptation
The aim of domain adaptation algorithms is to establish a learner, trained on labeled data from a source domain, that can classify samples from a target domain, in which few or no labeled data are available for training. Covariate shift, a primary assumption in several works on domain adaptation, assumes that the labeling functions of source and target domains are identical. We present a domain...
متن کاملSimultaneous Monitoring of Multivariate Process Mean and Variability in the Presence of Measurement Error with Linearly Increasing Variance under Additive Covariate Model (RESEARCH NOTE)
In recent years, some researches have been done on simultaneous monitoring of multivariate process mean vector and covariance matrix. However, the effect of measurement error, which exists in many practical applications, on the performance of these control charts is not well studied. In this paper, the effect of measurement error with linearly increasing variance on the performance of ELR contr...
متن کاملRobust Covariate Shift Prediction with General Losses and Feature Views
Covariate shift relaxes the widely-employed independent and identically distributed (IID) assumption by allowing different training and testing input distributions. Unfortunately, common methods for addressing covariate shift by trying to remove the bias between training and testing distributions using importance weighting often provide poor performance guarantees in theory and unreliable predi...
متن کاملAdaptive learning with covariate shift-detection for motor imagery-based brain-computer interface
A common assumption in traditional supervised learning is the similar probability distributionof data between the training phase and the testing/operating phase. When transitioning from the training to testing phase, a shift in the probability distribution of input data is known as a covariate shift. Covariate shifts commonly arise in a wide range of real-world systems such as electroencephalog...
متن کامل